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ABSTRACT 

This study aims to assess the impact of hypermedia designing as a cognitive tool on knowledge acquisition and 
retention in comparison to traditional instruction. A pretest-posttest control group design was used. The study 
included 48 second-year preservice computer teachers who enrolled in “Instructional Technology and Material 
Preparation" course at one of the universities in Turkey. The subjects were assigned to experimental and control 
groups through “matched-pair technique." Both groups were given a pretest to measure their prior achievement 
level in “Instructional Technology & Material Preparation” course content. While the control group was 
exposed to traditional instruction, the experimental group developed hypermedia on the course content in a 
constructivist learning context for 14 weeks. A posttest was given at the end of the treatment, and a retention test 
was given eight weeks after the treatment to both groups. The results showed that the use of hypermedia as a 
cognitive tool resulted in similar level of student achievement as traditional instruction. 
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INTRODUCTION 

Computers and other related technology systems such as multimedia/hypermedia, World Wide Web and virtual 
reality are used in the field of education through a variety of approaches. Revees (1999a) examines these 
approaches in two categories. In the first, students learn from interactive learning systems (computer-based 
instruction, traditional conception of use of computers) whereas in the second, students learn with interactive 
learning systems (cognitive tools). In the from approach, the content is encoded in the software programs, and it 
is assumed that communicating with this content, students will learn it. In the with approach, interactive tools are 
used by students as cognitive tools to analyze, access, interpret, organize, and represent information. As a result 
of this interaction, cognitive tools are expected to promote critical thinking and higher order learning in students. 

The computer technologies as cognitive tools represent a significant departure from traditional conceptions of 
technologies. In cognitive tools, information or solutions of problems are not encoded in the educational material 
to be learned, which are designed to transmit the knowledge to the learners effectively. With cognitive tools, 
learners function as designers and problem solvers in using the computers as tools for analyzing, accessing, 
interpreting and organizing their personal knowledge. In this process, computers become cognitive or mental 
tool helping learners’ thinking in performing the learning task (Jonassen, 2000; Kozma, 1992; Mayes, 1992; 
Reeves, 1999a; Toomey et al., 1995). Jonassen, Carr and Yueh (1998), and Reeves (1999b) state that when 
computers are used by learners to represent what they know, this procedure necessarily engages them in critical 
thinking and higher order learning about the content they are studying. In other words, they require students to 
think about what they know, and learn in different and meaningful ways. 

Hypermedia interactive systems can be used in both to learn from and to learn with (cognitive tools) approaches. 
The literature points out that when students learn from hypermedia, multiple representational formats such as 
combination of text, video, graphics, pictures and animations in a content domain through multimedia contribute 
to students’ learning, self competence and motivation (Kafai, Ching and Marshall, 1997). However, learning 
from hypermedia is not problem free. For example, it may not be effective in supporting learning; learners may 
not always select the right link for effective learning; it may be inefficient if there is no guide; engagement and 
interactivity which is essential for learning may not be provided in an effective way; discovery learning may be 
questionable through browsing; and lastly disorientation and cognitive overload may create problems in learning 
through hypermedia (Mayes, 1993). 

According to Jonassen and Reeves (1996), Jonassen (2000), and Dalgamo (2001), the use of hypermedia as a 
cognitive tool places students in the author’s seat so that they may construct their own understanding, rather than 
interpreting the teacher’s understanding of the environment, and they suggest that it is more efficient using 
hypermedia not as a source of knowledge to learn from but rather as a cognitive tool to construct and to learn 
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with. They argue that knowledge acquisition in the design can be facilitated when learners are actively engaged 
in designing knowledge rather than interpreting and decoding it. Learners become designers when they focus on 
the purpose of acquiring information, hypermedia’s underlying structure, and using the arguments required by 
the subject matter to justify the design. Hypermedia can be a powerful tool for engaging and supporting these 
activities. Therefore, providing appropriate learning environments for learners to design and present their own 
understanding of the subject through designing hypermedia can be an effective learning experience (Reeves, 
1999a). In addition, based on their research study, Kafai, Ching and Marshall (1997) indicate that designing and 
creating hypermedia is a complex task and most of the time requires collaboration and teamwork, and therefore 
using hypermedia as a cognitive tool promotes students’ collaboration and project management skills. 
Accordingly, Revees (1999a) acknowledged that “using multimedia construction programs as cognitive tools 
engages many skills in learners such as: project management skills, research skills, organization and 
representation skills, presentation skills, and reflection skills” (paragraph 7). 

Jonassen (1998) and Revees (1999b) state that using technology as cognitive tool and learning with technology 
in effective and efficient ways require constructivist-learning context. “Constructivism is based on the 
fundamental assumption that learning is in the activity of the learner. Learning is in the doing” (Duffy and Orrill, 
2001, p.l). They add that the teacher’s role in this approach is to provide appropriate learning environments and 
guide and facilitate for learning. As it is cited in Oliver and Herrington (2003), Cunningham, Duffy and Knuth 
(1993) indicated that constructivist learning contexts provide experience, realistic contexts, social experience 
appreciation for multiple perspectives and multiple modes of representation, promote ownership, and encourage 
self-awareness in the knowledge formation process. Constructivist approach can be implemented best at advance 
level learning (Jonassen, Mayes and McAlessi, 1993). They indicate that in the learning process there are three 
main phases: introductory, advanced and expert learning phases. 

Introductory learning occurs when learners have little directly transferable prior knowledge about a skill 
or content area. It represents the initial stages of schema assembly and integration. At advanced learning 
phase, learners acquire more advanced knowledge in order to solve more complex domain or context- 
dependent problems. Experts have more internally coherent and more richly interconnected knowledge 
structure (p. 1). 

They declare that initial knowledge acquisition phase is better served by classical instructional design while 
constructivist learning environments are generally more viable for the second, advanced knowledge acquisition 
phase. Universities and secondary education institutions exist to foster advanced knowledge acquisition with a 
transition to constructivist approach that represents complexity and ill-structuredness as the learner acquires 
more knowledge (Jonassen, Mayes and McAlessi, 1993). It can be concluded from these statements that using 
hypermedia as a cognitive tool in a constructivist learning context can be facilitated better at upper secondary 
and university education levels. 

The literature indicates that use of hypermedia as a cognitive tool is effective for critical thinking, higher order 
learning, project management skills, reflection skills, and presentation skills (Jonassen, 2000; Jonassen & 
Reeves, 1996; Lehrer, 1993; Mayes, 1993). Although numerous research studies offered evidence on the positive 
impact of hypermedia as cognitive tools on veriey of students’ skills, the literature seems to be insufficient in 
explaining the impact of hypermedia on knowledge acquisition and retention when hypermedia is used as a 
cognitive tool in a constructivist learning context at teacher education level. Since teachers are the ones who 
integrate technology into teaching and learning process, how hypermedia as a cognitive tool contributes to 
knowledge acquisition and retention in educational process needs to be explored at teacher education level as 
well. Additionally, in Turkey, the Higher Education Council (HEC) redesigned the curricula of teacher- 
education institutions to improve their quality, and to integrate information and communication technology 
(ICT) into these programs in 1998 (HEC, 1998). However, integration of ICT into teacher education programs 
has been implemented mostly at computer literacy level. Therefore, this study aims to assess the effect of use of 
hypermedia when it is used as a cognitive tool in comparison to traditional classroom instruction on knowledge 
acquisition and retention of preservice computer teachers. The following research questions guided this study. 

(1) Is there a significant mean difference in knowledge acquisition between the experimental group who used 
hypermedia as a cognitive tool and the control group who were exposed to traditional instruction? 

(2) Is there a significant mean difference in knowledge retention between the experimental group who used 
hypermedia as a cognitive tool and the control group who were exposed to traditional instruction? 
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METHOD 

Design 

A pre-test/post-test control group design was used in this study. Control variables of the study were prior 
achievement, gender and type of high school the participants graduated from; the independent variable was the 
treatment (hypermedia learning environment vs. traditional instruction); the dependent variables were the post¬ 
test and retention test performance of the subjects. Below, the details on the participants, the instrument and the 
procedures of the study are presented. 

Participants 

The study design included a total of 48 second-year Computer Education & Instructional Technology Education 
students who enrolled in “Instructional Technology and Material Preparation” course at one of the universities in 
Turkey, and lasted 14 weeks (two hours a week). The students in the Department of Computer Education & 
Instructional Technology Education are trained to be computer teachers for K-12 schools and experts in the field 
of instructional technology. Majority of the students graduated from computer departments of vocational or 
technical high schools. The students had the basic knowledge in variety of computer applications and in 
designing hypermedia prior to the experiment. They were assigned to experimental and control groups through 
“matched-pair technique” based on gender, cumulative GPA scores and the types of high schools they graduated 
from. From each pair, one student was assigned to the experimental group, the other was assigned to the control 
group randomly. By using this technique, equal representation in terms of gender, prior achievement, and prior 
education in both control and experimental groups was established. In the beginning of the study, the 
experimental group included 28 (24 males and 4 females), and the control group included 28 (24 males and 4 
females) subjects. The distribution of males and females in these groups represents closely the distribution by 
gender in the department. During the experiment, four students in the experimental group did not participate in 
the majority of the activities, therefore they, together with their pairs in the control group (total of eight 
students), were eliminated from the analysis of the data. As a result, 24 (20 males and 4 females) students in the 
control group, and 24 (20 males and 4 females) students in the experimental group participated in all phases of 
the study. 

Data collection instrument 

An achievement test was used to assess the students’ achievement on “instructional material development” 
subject before the experiment, after the experiment, and eight weeks after the experiment. The achievement test 
was developed by the researcher, and consisted of 44 multiple-choice questions on “instructional material 
development” subject. First, a table of specifications on the basis of learning objectives stated in the textbook 
Instructional Media and Technologies for Learning (Heinich, Molenda, Russell, & Smaldino, 2002) used in this 
course was constructed. Based on the table of specifications, the questions were written to establish the content 
validity of the test. So that lower and higher cognitive levels such as knowledge (recalling the learned subject), 
comprehension (understanding the meaning of the subject), application (using the learned subject in new 
situations) and analysis (understanding the underlying structure of the subject) were included in the achievement 
test. After the test was developed, two experts (a test construction specialist and a subject area specialist) 
assessed the test in terms of validity and relevance to the subject matter. Based on their suggestions the test was 
revised. After revisions, the test was piloted with a group of students (n=29) who had been taken the same 
course in the previous semester to check whether the items were clear and understood correctly. Afterwards, an 
item analysis was carried out on the test. The results of this analysis proved that one of the items in the test was 
too difficult for the subjects. Since the course content related to this specific item was new in the course and in 
the textbook used in the course, the item was not eliminated from the test. The Cronbach alpha reliability of the 
questionnaire was .78. 

Procedures 

The procedures and data analysis used in this study are presented in Table 1. At the beginning of the study, both 
experimental and control groups were given an achievement test to assess their prior achievement level in the 
course content. 
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Table 1: Data collection and analysis procedures 


Groups 

Pretest 

Treatment 

Posttest 
(given at the 
end of 
treatment) 

Retention test 
(given 8 
weeks after 
treatment) 

Data 

analysis 

Control 

Achievement 

Traditional 

Achievement 

Achievement 


(24 students) 

test 

classroom 
instruction 
(14 weeks) 

test 

test 

Means, 

standard 

deviations, 

Experimental 
(24 students) 

Achievement 

test 

Hypermedia as 
a cognitive tool 
(Hypermedia 
development by 
students on the 
course content, 

14 weeks) 

Achievement 

test 

Achievement 

test 

and t test. 


Then, students in the experimental group were informed of the procedures of the course that the course was 
going to precede in a constructivist context, and hypermedia was to be used as a cognitive tool. A constructivist 
learning environment in which students acted as designers of hypermedia learning environments was established 
to have learners work in groups through social interactions as suggested by Vygotsky (1978), and construct their 
own knowledge in the field of “instructional technology and material preparation.” To create constructivist 
learning environment, situated learning environment guidelines recommended by Herrington & Oliver (2000) 
were considered. By creating a hypermedia learning environment on the course content for preservice computer 
teachers, authentic context and activities were facilitated. Students had acces to the expert knowledge through 
weekly discussions with the instructor on the hypermedia content and template. A reflective environment was 
created through writing weekly reflective journal on the hypermedia content, receiving feedback on these 
journals, and discussing within groups both the content and design of the hypermedia. Through out the process, 
the group members had variety of roles such as instructional designers, programmers, project managers and etc. 
Through out this collaborative process, the instructor provided guidance for the students. 

Throughout the semester, the students in groups had to develop hypermedia as an instructional material in 
“instructional technology and material preparation” subject for other preservice teachers, and they covered most 
of the course content in that material. The instructor did not present the course content to the students 
didactically, but promoted a constructivist-learning context in which she facilitated, coached and guided. During 
the experiment, in order to guide the students effectively, predetermined stages, namely planning, content 
formation, template design, hypermedia development, evaluation, and presentation were followed by the 
students. In the beginning of the semester, the students in the experimental group perfonned planning activities. 
In planning they were requested to form their project groups for two to five students. In the groups, they 
determined the tasks they need to perform throughout the hypermedia construction process, negotiated each 
group member’s roles, and made task schedules to be followed in the hypermedia development process. As a 
result of their planning activities each group handed a group contract to the instructor. After planning, they 
started content formation of hypermedia learning environment. To determine the content of the hypermedia, the 
students wrote weekly reflective journals about the assigned subjects of the course content for about eight weeks, 
and received weekly feedback from the instructor on their journals. Based on the instructor’s feedback, the 
groups made some revisions on their hypermedia content. While preparing their journals they used the textbook. 
World Wide Web, assigned resources from the university library, and related materials from other courses. 
During this period, they also participated in group discussions and activities held in class. This process helped 
them investigate the subject area deeply and decide on which information to include in the hypermedia. Parallel 
to content formation process, they designed the templates of the hypermedia they were to develop. While 
developing their hypermedia learning environment template, each group’s template was discussed in class in 
terms of effective design principles, and each group had their classmates and instructor’s assessment of their 
template design. In this process they decide on the structure of the hypermedia, how to present information, how 
to link nodes. After they finished content fonnation and template design, by the end of the semester they 
developed the hypermedia learning environment. The groups used MS FrontPage, Java Scripts, Flash, PHP, 
and other similar tools to develop their hypermedia projects. Then they evaluated the hypermedia they 
developed to check if it is running properly without any problem, and made revisions. At the end of the semester 
they presented their hypermedia in class to share their projects with others. 
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The control group continued learning the “Instructional Technology and Material Preparation Course” content 
with regular classroom instruction mostly based on PowerPoint presentations supported by question-answer 
strategy (traditional instruction) while the experimental group developed hypermedia about the course content. 
The treatment continued 14 weeks. At the end of the treatment, the achievement test used as pre-test was given 
to the students in both groups as post-test. Eight weeks after the experiment, the same achievement test was 
given to both groups once again to measure the level of knowledge retention. The data collected through pretest, 
posttest and retention test were analyzed by descriptive and inferential statistics such as means, standard 
deviation and t-test. 

RESULTS 
Pretest results 

Pretest results showed that in the beginning of the experiment the experimental group who used hypermedia as a 
cognitive tool had higher scores from the control group that was subjected to traditional classroom instruction 
(M=15.04 and 12.75 respectively) (see Table 2). However, t-test results indicated that the difference in the 
pretest mean score was not statistically significant (p>.05). 


Table 2: Pretest mean score results of experimental and control groups 



N 

Mean 

Std. Deviation 

Experimental Group 

24 

15.0417 

4.0269 

Control Group 

24 

12.7500 

4.3564 


t(46)=l.89, p=.065 


Posttest results 

As it is presented in Table 3, posttest results showed that at the end of the experiment, students in the 
experimental group who were exposed to hypermedia as a cognitive tool scored higher than the students in the 
control group who participated in traditional classroom instruction (M=36.83 and 35.79 respectively). However, 
t-test results indicated that the difference in the postest mean score was not statistically significant (p>.05), (see 
Table 3). 


Table 3: Posttest mean score results of experimental and control groups 



N 

Mean 

Std. Deviation 

Experimental Group 

24 

36.8333 

4.0931 

Control Group 

24 

35.7917 

7.0955 


t(46)=.62, p=.53 


Retention test results 

Retention test results (eight weeks after the experiment) showed that the students in the experimental group 
scored slightly higher than the students did in the control group (M=29.79 and 28.41 respectively; Table 4). 
However, t-test results indicated that the posttest-retention test gained scores mean difference of experimental 
and control groups (M= -7.04 and -7.37 respectively) was not statistically significant (p>.05), (see Table 5). 


Table 4: Retention test mean score results 



N 

Mean 

Std. Deviation 

Experimental Group 

24 

29.7917 

3.7181 

Control Group 

24 

28.4167 

6.1285 

Table 5: Posttest-retention test gained-score means of experimental and control groups 


N 

Mean 

Std. Deviation 

Experimental Group 

24 

-7.0417 

4.4573 

Control Group 

24 

-7.3750 

4.9150 


t(46)=.24, p=.80 


CONCLUSION 

Posttest and retention test results showed that the second year preservice computer teachers in both experimental 
and control groups acquired and retained similar levels of knowledge in the area of “instructional technology and 
material development.” It can be concluded from these results that the two approaches do not differ from each 
other significantly in regard to knowledge acquisition and retention. Literature indicates that using hypermedia 
as a cognitive tool helps learners analyze, access, interpret and integrate information, and facilitate reflective 
thinking, critical thinking and higher order learning (Jonassen, 2000; Jonassen & Reeves, 1996; Lehrer, 1993; 
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Mayes, 1993). To achieve all these skills, of course, one must have the knowledge base in the content area. So 
the assumption would then be that hypennedia, as a cognitive tool would contribute to knowledge acquisition 
and retention more effectively than the traditional instruction. The results of this study prove otherwise. 

Several explanations can be offered in interpreting the results of this study. One of the reasons for this result may 
be related to the learning and study habits of the participants. The participants of this study were the second year 
preservice teacher education students in Turkey. Turkey’s educational system is quite traditional. Throughout 
their school years, students might have received direct instruction most of the time. The students might have 
carried these study habits from the past, and developing hypennedia about the subject to be learned might be a 
new learning approach for them. The demanding instruction/leaming process and in adapting to this new 
strategy might have negatively affected their knowledge acquisition and retention. 

This study lasted 14 weeks, which was a long period of time. Duration of the project was consistent with the 
literature stating that through project-based learning, learners concentrate on complex, integrated modules of 
long-tenn instruction (Krajcik et al., 1994 cited in Jonassen 1998). However working in a group for a long 
period of time was a new experience for the majority of the students in the experimental group. This might have 
also affected their perfonnance negatively in the post-test and retention test. 

This study provides valuable infonnation as to whether designing hypennedia as a cognitive tool in comparison 
with traditional classroom instruction leads to learning and to what degree. Based on the results, it can be 
suggested that hypermedia can be integrated into “Instructional Technology and Material Preparation” course as 
a cognitive tool, to provide practice opportunities in instructional material development process for prospective 
computer teachers. The results of this study may help practitioners gain perspective into using hypennedia as a 
cognitive tool in their classes, and understand what to expect in terms of knowledge acquisition and retention in 
teacher education context. The results of this study may also help instructional material designers/developers 
and those who use the material in educational settings in understanding the potential contribution and limitation 
of hypennedia as a cognitive tool in a constructivist learning context. 

Jonassen (2000) states that using computer-based cognitive tools promotes critical thinking skills of the students. 
However, he indicates that it may not be possible to observe these thinking skills of the learners immediately 
after the application. It can be observed in later stages of their life when students face with a problem situation. 
In addition, the students might have gained other skills such as collaboration, problem solving, and project 
management skills as Kafai, Ching and Marshall (1997) indicated. 

In order to interpret the findings of this study, some of the limitations need to be taken into account. In this 
study, small sample size was the most important limitation of the study. The number of the participants in the 
study was limited to the number of second year students at Instructional Technology & Computer Education 
Department. The students had the necessary knowledge and skills in hypennedia development, and they did not 
get any orientation or training in hypermedia development area. Therefore one needs to be cautious in 
generalizing the results of this study. Despite these limitations, this study provides valuable contribution with 
regard to relationship between knowledge acquisition and retention, and use of hypennedia as a cognitive tool in 
teacher education institutions. 

In further studies, in addition to the knowledge acquisition and retention, investigations to assess other skills and 
gains of the students from use of hypennedia as a cognitive tool can be suggested. It is recommended that to be 
able to see the contribution of hypermedia for learning as a cognitive tool, further research studies in different 
subject area, with different learners groups, and with different time periods are needed. 
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